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TO: Board Members 


FROM: Terry B. Grier, Ed.D. 
Superintendent of Schools 


SUBJECT: THE EFFECTS OF THE LITERACY BY 3 INITIATIVE ON THE READING 
ACHIEVEMENT OF HOUSTON INDEPENDENT SCHOOL DISTRICT (HISD) 
FIRST- TO FIFTH-GRADE STUDENTS, 2014-2015. 


CONTACT: Carla Stevens, 713-556-6700 


Literacy By 3 is a comprehensive, literacy initiative implemented in HISD’s kindergarten through 
fifth-grade classrooms during the 2014-2015 school year. Literacy By 3 is a combination of six 
reading strategies: (1) guided reading, (2) reading to self (independent or silent sustained 
reading), (3) read to partner, (4) read to class (read-aloud), (5) word work (vocabulary), and (6) 
writing. The HISD curriculum is the foundation for Literacy By 3, which is based on the Texas 
Essential Knowledge and Skills (TEKS), College and Career Readiness Standards (CCRS), 
English Language Proficiency Standards, and the Prekindergarten guidelines (PKG). The 
purpose of this evaluation was to determine the extent to which proposed Literacy By 3 
instructional strategies were adopted and implemented across HISD classrooms. This 
evaluation also measured Literacy By 3 effects on the reading performances among first- 
through fifth-grade HISD students during the 2014—2015 school year as well as teacher 
perceptions and experiences. 


Most of the classrooms visited were organized and arranged for Literacy By 3 and followed all 
six Literacy By 3 instructional strategies. Pre and posttest results showed that third-, fourth-, and 
fifth-grade students performed better in 2015 than they did in 2014 on the State of Texas 
Assessments of Academic Readiness (STAAR) reading tests. The results were statistically 
significant (p < .0001). Texas Education Agency (TEA) has determined that for accountability 
purposes, individual fourth-grade students with a scale score difference of 82, over the same 
period, would have Met Progress on reading in 2015. The difference for a fifth-grade student 
would have been 32 scale score points. 


Most of the 701 teachers surveyed either agreed or strongly agreed that the initial training, 
campus leadership support, and follow-up training were effective in the implementation of 
Literacy By 3, and that their teaching practices were consistent with Literacy By 3 instructional 
strategies. Implications include additional training in key aspects of Literacy By 3, particularly, 
use of running records, small group organization, and exposure to best practices and exemplars 
in Literacy. 
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Executive Summary 


Literacy By 3 is a comprehensive, literacy initiative implemented in HISD’s kindergarten- through fifth- 
grade classrooms during the 2014-2015 school year. Literacy By 3 is a combination of six reading 
strategies: (1) guided reading, (2) reading to self (independent or silent sustained reading), (3) read to 
partner, (4) read to class (read-aloud), (5) word work (vocabulary), and (6) writing. The HISD curriculum is 
the foundation for Literacy By 3, which is based on the Texas Essential Knowledge and Skills (TEKS), 
College and Career Readiness Standards (CCRS), English Language Proficiency Standards, and the 
Prekindergarten guidelines (PKG) (HISD, 2014). The ultimate objective of Literacy By 3 is to have, by 2019, 
all HISD students reading on grade level by the end of third grade. Much of the focus, therefore, has been 
on the kindergarten to third grades. This focus was in direct response to the slow growth in the district’s 
reading scores on various standardized assessment, including the State of Texas Assessments of 
Academic Readiness (STAAR), Stanford 10, and the National Assessment of Educational Progress (NAEP) 
in 2013-2014. It was due also to the critical role reading by third grade plays in later reading and academic 
achievement (Annie E. Casey Foundation, 2010). This study was designed to evaluate the effects of the 
inaugural year of the Literacy By 3 initiative on the reading performances of first- through fifth-grade HISD 
students. 


Highlights 

e Paired t-test results showed that third- through fifth-grade students in the sample had higher 
STAAR mean reading scale scores in 2015 compared to 2014. 

e The Literacy By 3 initiative resulted in important to substantially important effect sizes (eta-squared 
statistic between 0.22 to 0.50) in reading, which is equivalent, on average, to two — seven months 
reading improvement at the third through fifth grades. 

e When disaggregated by ethnicity and economic status, the effects on reading were important to 
substantially important at all three grade levels. 

e A higher proportion of students met Level II Satisfactory phase-in 1 in third grade for retained 
students and in fifth grade and Advanced reading performance standards at all three grade levels 
on the 2015 STAAR after one year of Literacy By 3 instruction. 

e Over 53% of first-grade students scored at or above the 50 NPR on the 2015 lowa ELA and 
reading assessments. 

e Teacher perceptions and experiences of key aspects of Literacy By 3 were on average rated 
between 3.52 and 4.18 on a 5-point Likert scale. 

e All classrooms observed were organized in accordance with Literacy By 3 specifications. Most 
teachers tracked students’ performance and taught using Literacy By 3 strategies. 

e About one-third of schools did not allocate the recommended 135 daily instructional minutes for 
delivering Literacy By 3. 


HISD Research and Accountability 1 


Recommendations 


Consistent with respondents’ recommendations, the district should continue training in small group 
organization and instruction, guided reading, instructional strategies, and running records for 
Literacy By 3 in order to improve classroom practices. 

Continued training in the management of small or guided reading groups should focus on how to 
keep students engaged, how to pace work assignment for students, and exposing teachers to 
exemplars and Literacy By 3 best practices. The Leveled books and related materials for all, and 
in particular limited English proficiency (LEP) classrooms, need to be available to support 
instruction. 

Students should be exposed to the required 135 minutes for Literacy By 3 instruction. 


Administrative Response 

The Elementary Curriculum and Development Department is very pleased that third-grade STAAR reading 
served as a bright spot coinciding with the first year of Literacy By 3. HISD saw a two-percentage point 
increase, which was higher than the state with only a one-percentage point gain demonstrating promise 
and future impact. Additionally, HISD saw promising results on the lowa assessment with over 53% of first- 
grade students scoring at or above the 50'" NPR on the 2015 reading assessment. 


In response to the overall third- through fifth-grade STAAR reading results, Elementary Curriculum and 
Development will implement the following actions to support campuses and increase student achievement: 


Continue to provide principal and teacher training to support Literacy By 3 with an emphasis on 
comprehension of complex text to think deeply and critically aligned to the rigor of the STAAR 
Reading. 
Evaluate STAAR results and identify elementary campuses with significant gains to determine 
elements that contributed to their success. There are over two dozen elementary campuses with 
double-digit increases in STAAR reading this past school year. 
Provide pre-kindergarten (pre-k), fourth- and fifth-grade classroom libraries, and kindergarten 
through second-grade Spanish language materials along with teacher training to expand the 
success of Literacy By 3. 
Partner with the Multilingual department to create systems, practices, and supports for students 
transitioning from Spanish to English 
Partner with Elementary School Offices to create networks in order for principals and teachers to 
share best practices, look at student work, analyze formative assessment data, plan lessons 
grounded in the district curriculum, and collaborate with teachers on demonstration campuses. 
Teacher Development Specialists and School Support Officers will leverage school leaders and 
teacher leaders on demonstration campuses in the following ways: 

e Instructional coaching 

e Peer observation 

e Formative assessment collaboration 

e Data-driven instruction 

e Collaborative lesson planning 
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Introduction 


Literacy By 3 is a comprehensive, literacy initiative implemented in HISD’s kindergarten- through fifth- 
grade classrooms during the 2014-2015 school year. Literacy By 3 is a combination of six reading 
strategies: (1) guided reading, (2) reading to self (independent or silent sustained reading), (3) read to 
partner, (4) read to class (read-aloud), (5) word work (vocabulary), and (6) writing. The HISD curriculum is 
the foundation for Literacy By 3, which is based on the Texas Essential Knowledge and Skills (TEKS), 
College and Career Readiness Standards (CCRS), English Language Proficiency Standards, and the 
Prekindergarten guidelines (PKG) (HISD, 2014). The Literacy By 3 initiative is considered a balanced 
approach to literacy that includes phonics, vocabulary, fluency, and comprehension as recommended in 
the 2001 National Reading Panel Report (see Moats, 2010). The belief is that reading instruction is 
connected to writing, listening, and speaking, and that reading is comprised of decoding, fluency, language, 
and text comprehension (HISD, 2014). It is the belief as well that each reading component must be explicitly 
taught and practiced to mastery; and that evidence-based practices and established standards must guide 
reading instruction (HISD, 2014). Literacy By 3 appears to support these beliefs. 

The ultimate objective of Literacy By 3 is to have, by 2019, all HISD students reading on grade level by 
the end of third grade. Much of the focus, therefore, has been on the kindergarten to third grades. This 
focus was a direct response to the slow growth in the district’s reading scores on various standardized 
assessments, including State of Texas Assessments of Academic Readiness (STAAR), Stanford 10, and 
the National Assessment of Educational Progress (NAEP) in 2013-2014. It was due also to the critical role 
reading by third grade plays in later reading and academic achievement (Annie E. Casey Foundation 
(2010). 

The program’s implementation included a professional development component delivered through 
several summer sessions as part of the School Leadership Institute and supported through early-release 
days and afterschool follow-up sessions. Teachers received campus-based support from literacy 
managers, teacher specialists, teacher development specialists, and school support officers. This support 
included classroom visits and walkthroughs with feedback to teachers and school administrators. The 
initiative also produced “caught-in-the-act” videos of best Literacy By 3 practices as exemplars on the HISD 
website. 

The purpose of this evaluation was to determine the extent to which proposed Literacy By 3 instructional 
strategies were adopted, implemented, and changed reading instruction across HISD classrooms. This 
evaluation also measured the Literacy By 3 effect on the reading performance among first- through fifth- 
grade HISD students during the 2014-2015 school year. Specifically, the evaluation focused on teacher 
experiences and perceptions, the extent to which proposed Literacy By 3 practices were adopted, and the 
effect of the initiative on students’ reading performance using the STAAR and lowa Assessments results. 

The evaluation answered the following questions: 

(1) How were teachers and school leaders prepared for the implementation of Literacy By 3? 

(2) To what extent did the walkthrough classrooms reflect Literacy By 3 practices? 

(3) What were the experiences and perceptions of teachers who implemented Literacy By 3? 

(4) What were the effects of the Literacy By 3 initiative on third- through fifth-grade STAAR? 

(5) How did the Literacy By 3 first- through fifth-grade students perform on the lowa Assessments? 


Literature review 
Professional Development 
In-service teacher professional development associated with specific program implementation has 
been linked to improvements in student learning outcomes (Yoon, Duncan, Lee, Scarloss, & Shapley, 
2009). Teachers who received substantial professional development, 49 hours across nine studies 
reviewed, boost their students’ achievement by 21 percentile points. Studies that had more than 14 hours 
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of professional development supported by follow-up sessions showed positive and significant effects on 
student achievement from professional development (Yoon, Duncan, Lee, Scarloss, & Shapley, 2009). 

Teachers who participated in an eight-day professional development reading institute and seminar 
scored higher with effect sizes of 0.37 and 0.38, respectively, on a related teacher knowledge test than did 
teachers in the control schools (Garet, et al., 2008). In addition, both the institute and seminar teachers 
used significantly more explicit reading instruction compared to teachers in the control group. However, 
there were no statistically significant differences between the reading achievement of students whose 
teachers participated in the institute and seminar, and the control group (Garet, et al., 2008). 


Direct Reading Instruction 

Direct or explicit reading instruction involves the impartation of new information to students through 
meaningful teacher interactions and teacher guidance of student learning with the teacher leading the 
teaching-learning process. It involves explicit explanations, modeling or demonstrating, and guided practice 
(Rupley, Blair, & Nichols, 2009). Direct or explicit reading is recommended as an integral part of learning 
the major content strands of the reading process: phonemic awareness, phonics, fluency, vocabulary, and 
comprehension (Rupley, Blair, & Nichols, 2009). 

A study of fourteen second-grade students and teachers found that students with reading difficulties 
had limited opportunities through core reading instructions and supplemental reading interventions to 
receive research-based, effective vocabulary instruction (Wanzek, 2014). Only 8% of the core classroom 
reading was devoted to direct vocabulary instruction. Minimal amounts of direct vocabulary instruction 
occurred in supplemental reading instructions (Wanzek, 2014). 


Guided Reading 

Guided reading is “an instructional approach that involves a teacher working with a small group of 
students who demonstrate similar reading behaviors and can all read similar levels of texts, which are easy 
enough for students to read with teachers’ skillful support” (Scholastic Inc. , 2014, p. 28). The goal of guided 
reading is to have students read a variety of leveled text with ease, that is, with 90% accuracy, and deep 
understanding (Scholastic Inc., 2014). This approach assumes that students understand how print works, 
will apply reading strategies of their own, know how to monitor their own reading, and have the ability to 
search for possibilities and alternatives if they encounter problems when reading (Scholastic Inc., 2014). 

Guided reading has resulted in significant improvement in the reading achievement of struggling 
readers (Green, 2010), second-grade reading comprehension (Lorent Deegan, 2010), and overall reading 
achievement (Green, 2010). There was a significant correlation between guided reading instruction and 
improvement on the Illinois Standards Achievement Test scores when students were tested over time 
(Underwood, 2010). Time appeared to have been a significant factor in the effectiveness of guided reading 
programs; and longer implementation appeared to have larger effects on student reading performance 
(Green, 2010; Underwood, 2010). Bruce (2010) did find no signifcant difference in reading scores before 
and after fourth-grade reading instruction among at-risk students in one southeastern U.S.A. Title 1 school. 


Read-Aloud 

Read-aloud is a strategy in which time is set aside to read to students orally from text that are at their 
listening level but above their independent reading level, that is, 90% understanding and 95% accuracy 
(Beltchenko, n.d.). Increasingly, student-to-student and community volunteers are used during classroom 
read-aloud. 

Read-aloud has been associated with increased reading enjoyment for students resulting in increase 
in the times students spend reading silently (Pregg & Bartelheim, 2011). Students’ active engagement in 
read-aloud has been associated with greater vocabulary development (Beck & McKeown, 2001). Teacher 
read-aloud has been shown to correlate with comprehension development, growth in background 
knowledge, listening skills, and attitudes to reading among elementary students (Baker, Chard & Edwards, 
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2002; Fisher, Flood, Lapp & Frey, 2004; Santoro, Chard, Howard & Baker, 2008, (Lorent Deegan, 2010). 
Reading aloud to children and restricting their online time correlated with frequent reading, that is, reading 
for fun at least five times a week (Scholastic, Inc, 2015). 


Independent Reading (Read to Self) 

Independent reading is the level at which students read fluently with 90% or higher comprehension and 
95% word recognition (Burns & Roe, 2002; Johns, 2005). It involves student’s alone time for sustained 
silent reading. The assumption is that students develop reading skills by reading constantly. Independent 
reading has been described as the end product of a balanced literacy curriculum, where students are 
allowed to apply and practice strategies they have learned during guided reading, shared reading and read- 
alouds. Using materials appropriate to their levels, students demonstrate they are able to read confidently 
on their own and are excited about their reading abilities (Marshall, 2014). 


Word Work (Vocabulary) 

Word knowledge or a large, rich vocabulary and the skills for using those words have been described 
as the greatest tools we can provide students for succeeding in their education and in life (Pikulski & 
Templeton, 2004). The National Reading Panel Report (2000) affirmed the long standing importance of 
vocabularly knowledge in the development of reading and that since 1924, research had shown that the 
growth of reading power relies on the continuous growth of word knowledge. 

A two-year study of 90 British school children found that letter knowledge and phoneme sensitivity 
predicted word recognition skills. Vocabulary knowledge and grammatical skills predicted reading 
comprehenison (Muter, Hulme, Snowling, & Stevenson, 2004). Results from structural equation modeling 
on independent samples of fourth to eight-grade students found that vocabularly knowledge was a constant 
and significant predictor of overall reading comprehension, irrespective of grade level (Yovanoff, Duesbery, 
Alonzo, & Tindal, 2005). 

A study of 203 third-grade students using confirmatory factor analyses, structural equation modelling, 
and hierarchical regression analyses showed a stronger relationship between vocabulary breath over depth 
and reading comprehension, however both factors overlapped on predictability for reading comprehension 
(Tannenbaum, Torgensen, & Wagner, 2006). 


Literacy By 3 

Literacy By 3 brings togther five discrete literacy components, direct instruction, guided reading, read- 
aloud, independent reading, and word work, using leveled books with age- and grade-appropriate texts. 
These texts provided scaffolded instruction for children as a foundation for the acquisition of essential 
literacy skills, like fluency and comprehension. 

The teacher becomes a facilitator that provides students with reading strategies that guide their 
understanding through a process of inquiry using effective instructions and strategies that most engage 
students and matching resources to meet students needs and interests (Laquinta, 2006 cited in Scholastic 
2014). Teachers decipher for students the story behind the facts presenting them as useful and worth 
understanding (Wiggins and McTighe in Wilhelm, 2004 cited in Scholastic, 2014). This is best achieved by 
using flexible, supportive, small-group approaches that builds a community of readers through 
improvisation and a process of guided reading (Scholastic Inc., 2014). Notwithstanding, teachers’ 
knowledge of students and the instructional way in which they connect to readers with text information is 
pivotal to the success of the Literacy By 3 initiative and are premised on the teachers’ willingness to take 
time to survey students’ interest, background and sociocultural identities (Scholastic, Inc, 2014). This 
involves a balancing act that moves away from a whole-class approach to differentiation. 
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Method 


Data Collection and Analysis 

The data collection methods consisted of an online teacher survey, classroom walkthrough visits, and 
queries of HISD student assessment databases: Public Education Information Management System 
(PEIMS), STAAR, and lowa Assessments. These are detailed below. 


Teachers’ Survey 

Teachers were surveyed to determine their experiences and perceptions regarding the implementation 
of the Literacy By 3 initiative using multiple choice, Likert, and open-ended questions. The survey was 
tested and administered via SurveyMonkey™. SurveyMonkey™ is a free software used to collect, collate, 
and conduct descriptive analysis of online survey data. The survey was emailed to all teacher participants 
in the initiative. 

Likert-type questions were analyzed individually or were scaled and combined into composite scores. 
Likert-type items generate ordinal data while Likert scale questions generate interval data. The former is 
analyzed using descriptive data and the latter uses means and standard deviations as measures of central 
tendency and data variability (Boone & Boone, 2012; Clason & Dormody, 1994). 

In addition, Literacy By 3 classroom walkthroughs were conducted in fourteen first- and third-grade 
classrooms in seven elementary schools. Ten schools were selected using stratified random sampling 
based on the ranked percentage of students scoring at or above the 50' National Percentile Rank (NPR) 
on the 2013 Stanford 10 reading subtest. Schools were grouped in quartiles based on these ranks. The 
first school in each quartile was selected at random and every tenth school chosen thereafter formed the 
walkthrough sample. Seven schools were visited because of conflicts with assessment schedules and end- 
of-the-school-year activities. 


Walkthrough 

A walkthrough observation schedule based on the Scholastic Inc. Leveled Book Room implementation 
guide was developed. The schedule and summary of the observations are provided in Table 1 (Appendix 
A, page 22). Unique identification codes were used in this report to protect teacher, classroom, and school 
identities. Scholastic Inc. is the contracted vendor that provided the reading materials and program support 
for Literacy By 3. Literacy routines involving the five Literacy By 3 strategies occurred at the same time in 
all classrooms. Classes had to be observed while routines were already in progress. Observations lasted 
approximately one hour. Because the routines were not seen in their entirety, classroom artifacts, learning 
aids, and manipulatives that demonstrated use of key Literacy By 3 strategies were considered as evidence 
of program implementation. All walkthroughs were conducted in collaboration with Teacher Development 
Specialists, principals, or Literacy Specialists. Principals or assistant principals were debriefed after each 
visit. 


Assessment Data 

Analyses of third- through fifth-grade students’ performance on the 2015 STAAR reading test and the 
2015 lowa Assessments reading subtest were used to determine the average standard score and the 
proportion of students scoring at or above the 50* national percentile rank (NPR). The lowa Assessments 
are norm-reference tests administered to all HISD students in kindergarten through eighth grade. The 
2014-2015 school year marked the first administration of the test in HISD as it replaced the previously- 
administered Stanford Assessments. 

The test results were disaggregated by grade, ethnicity and students’ economic status. Students’ mean 
scale scores and the proportion who met STAAR satisfactory reading standards were used to determine 
the effect of Literacy By 3 on students’ reading performance using a repeated measures design. Repeated 
measures are based on before and after test scores of the same sample group to control on the extraneous 
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variables. Such control adds rigor and increases the probability that changes in students’ test scores would 
probably be attributed to the Literacy By 3 initiative, assuming that no other initiatives or programs were 
implemented, concurrently. The Texas Education Agency has adopted an effect size of 0.25 as a 
“substantially important” effect size for evaluating education program impacts (Texas Education Agency, 
2011). This will be the benchmark for evaluating the effect size of Literacy By 3 using the eta-squared 
statistic. Eta-squared statistic can be defined as the proportion of variance associated with or accounted 
for by each of the main effects, interactions, and error in an analysis of variance (ANOVA) study (Brown, 
2008). 


Sample 

A total of 28,047 students comprised the repeated-measures design sample (519 third-grade, 14,229 
fourth-grade, and 13,299 fifth-grade students). The third-grade students appeared to be repeaters or 
students who were in the third grade for at least two years. At the 95% confidence level and a confidence 
interval of five (5), this sample size yielded a sampling error of 0.53. The data set, disaggregated by grade, 
met the normality, homoscedasticity, and collinearity conditions using the Shaphiro-Wilk test, the normal 
Q-Q plot and the Detrended normal Q-Q plot on the IBM Statistical Package for Social Sciences (SPSS) 
software. 

The student population sample consisted of third- to fifth-grade students who had a reading score on 
both the 2014 and 2015 STAAR reading test, and all first- to fifth-grade students with a reading standard 
score and a NPR on the 2015 lowa Assessments. Students without a score were not included in the sample. 
STAAR regular English and Spanish versions of the exams were included in the sample. 

The teacher sample consisted of all prekindergarten to fifth-grade teachers. In total, 701 teachers 
completed the survey. With approximately 3,213 teachers in prekindergarten through fifth grade, this is a 
response rate of 21.8%. According to Resnick (2012), online surveys typically yield, based on anecdotal 
evidence, between a 0.5% and 1.5% response rate. Of the 701 teachers who completed the online survey, 
74.6% were English language instructors and 25.2% were Spanish language instructors. Of these, 20.5% 
were kindergarten teachers, 18.2% were first-grade teachers, 16.1% were second-grade teachers, and 
12.7% were third-grade teachers. Prekindergarten, fourth, fifth, and other unspecified grades were less 
than 9% each of the survey respondents. Teachers were either in self-contained (66.6%) or 
departmentalized classrooms (33.4%). 


Data Limitations 

Some schools were departmentalized with 90 instructional minutes of Literacy By 3 instead of the 
recommended 135 minutes. This may have affected students’ reading performance. Other programs like 
Time2Know and Read Houston Read may have been implemented concurrently with Literacy By 3, which 
could have compounded students’ reading performance. 

Classroom walkthroughs were conducted in the final two weeks of the school year. Due to end-of-year 
activities and testing, lessons observed may not have been as ‘routine’ as they could have been since 
arrangements had to be made to accommodate the timing of the observations. 


Results 
How were teachers and school leaders prepared for the implementation of Literacy By 3? 
According to 2014-2015 E-train data, 10,101 teachers participated in five professional development 
programs designed to familiarize and prepare teachers for delivering Literacy By 3 in all HISD targeted 


classrooms. The distribution of teachers and school leaders who participated in these preparation programs 
are shown in Figure 1 (see Table 2, Appendix A, page 24). 
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Figure 1. Percentage of Literacy By 3 summer professional development participants, 2014-2015. 


According to Figure 1, the four Literacy By 3 professional development activities were the “ABCs 
of Guided Reading”, “ABCs of Guided Reading Day1 and 2”, “Independent Reading”, and “Read 
Aloud’. 

The 2014—2015 E-train data also reflected that teachers and school leaders who completed the 
programs were exposed to two sessions, six hours each, of initial professional development 
activities related to the Literacy By 3 initiative. It is likely that teachers completed multiple programs 
and were probably exposed to more than twelve hours of professional development. 

In total, the initiative had 56 sessions, 6 or 12 hours each, except for the “ABCs of Guided Reading 
Day 1 and 2, of initial professional development. There were two sessions, two hours each, for the 
“ABCs of Guided Reading.” 

A majority of teachers, between 74.7% and 98.9%, completed each of these programs in which 
they enrolled. The “ABCs of Guided Reading, Day 1 and 2” had the highest participant completion 
rate of 98.9%. 

The other enrollees either did not complete the professional development, dropped out or did not 
show. 


To what extent did the walkthrough classrooms reflect Literacy By 3 practices? 


Of the seven schools where walkthroughs were conducted, two had improvement required (IR) status 
on the state accountability system; two met state standards with distinctions’, and three met standards in 
2014 (Table 3, Appendix A, page 25). State standards are determined using weighted scores on student 
achievement, student progress, closing the achievement gap, and postsecondary readiness. 


As shown in Table 1 (Appendix A, page 22), all of the first- and third-grade classrooms visited were 
arranged to Literacy By 3 specifications. There were leveled books, technology centers or corners 
containing computers and other audiovisual equipment (except in one classroom), desks were 
arranged for small group and guided reading. There were independent reading corners, teachers’ 
desks, rugs for read aloud and direct instruction, interactive white boards, and chalkboards for 
direct instruction. In addition, there were class libraries with books labeled according to reading 
levels and genres. 


1 Distinctions are awarded in reading/ELA, math, science, student progress, closing the achievement gap, 
and/or postsecondary readiness. 
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Small groups appeared to be the common method of instruction during visits with the exception of 
two classrooms in one of the seven schools. Teachers in those classes admitted that they struggled 
with the small groups and that they preferred whole-class instruction. 

Teachers in the classrooms observed appeared to struggle with two things: (1) managing the rest 
of the class during small-group guided reading, particularly those groups who needed support or 
assistance during independent work time, and (2) the challenge of outlining clear guidelines with 
adequate workloads for students during independent work time and during guided reading 
sessions. There were four classrooms that appeared to avoid these two struggles with excellence 
and these appeared to be exemplars of Literacy By 3. 

Guided reading, independent reading, word work, and read-aloud were observed in all of the 
fourteen first- and third-grade classrooms visited. Most teachers observed (12) appeared to track 
students’ performance using assessment tools such as Istation. Most classes had some form of 
tracking charts on display. They also appeared to have rotation schemes using color charts for the 
small-group instruction. 

Only two of the observed classrooms demonstrated active use of running records. This is not to 
suggest they were not used since the observation occurred during the end of the school year and 
at a single point in time. 


What were the experiences and perceptions of teachers who implemented Literacy By 3? 


Teachers were asked to rate their level of agreement or disagreement with fourteen statements related 
to key aspects of Literacy By 3 on a 1-5 Likert scale. These were collapsed into five key themes. The 
frequency of responses is summarized in Figure 2 (Table 4, Appendix A, page 27). 


Average Rating 
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Figure 2. Summary of teacher survey respondenis on key Literacy By 3 training effects, materials, 


and practices, 2014-2015. 


Teacher literacy practices had the highest average rating of 4.18 on a 5-point scale. The training 
effect on teacher perceptions of their understanding of what it means to be an effective literacy 
teacher had the lowest ratings with an average of 3.64. 

Material quality and usefulness had an average rating of 3.73. 

Monitoring performance and growth, which includes the use of running records and Istation data to 
monitor students’ performance and form guided reading groups, had a 3.73 average rating. 


Teachers were asked to state their agreement or disagreement to statements regarding their Literacy 
By 3 training on a 1-5 Likert Scale. Figure 3 shows the results. 
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Figure 3. Teacher perception and experiences with key Literacy By 3 training activities, 2014-2015. 


e The average ratings on all seven statements related to training activities were below 4.0, which is 
the “agree” rating on the 5-point scale (Table 5, Appendix A, page 28). 

e The following statements had average rating of 3.53: The information | learned during the four-day 
training prepared me to begin implementing Literacy By 3 in my classroom and the early release 
training on small group Guided Reading instruction provided learning that helped me to better plan 
and deliver small group instruction to my students. There was also 3.53 average rating for the 
statement, the running records training provided an opportunity to understand how to move 
students’ reading levels. 

e Campus-based support and information about best literacy practices recorded the highest average 
rating (3.75). 

Figure 4 shows teacher respondents’ perspectives on the link between the Literacy By 3 and HISD 

curriculum and instructional guides (Table 6, Appendix A, page 29). 
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Figure 4. Teacher perceptions and experiences with HISD instruction support framework and 
Literacy By 3, 2014-2015. 
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The average weighted scores for all six Likert-type statements were lower than 4.0. 

Teachers’ “use of the read aloud protocols as outlined in the HISD curriculum guides,” had the 
highest average rating (3.95) for instructional support framework, followed by the use of the “anchor 
charts” (3.90). 

The extent to which the structures of the HISD elementary reading curriculum planning guides 
facilitated the ease of planning received the lowest weighted average of 3.46. 


Teachers were also asked to identify additional details they would like to see as support for Literacy By 
3 implementation. Their responses were categorized into emergent themes. Themes were developed 
based on the initial reading of the responses and recategorized as new themes emerged to better capture 
the essence of those responses. Using a tally sheet, each response was collated by theme. Figure 5 
displays the additional support suggested with ten or more responses. 
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Figure 5. Teacher suggested additional support for Literacy By 3, 2014-2015. 


Based on Figure 5, teacher respondents proposed additional support in the following areas: training 
in organizing and instructing small groups (n = 127), running records (n = 47), other training (n = 
40), modeling lessons and best practices (n = 35), prekindergarten and kindergarten strategies (n 
= 26), ideas for workstations and centers (N = 23), dual language models (n = 21), material and 
resources (n = 13), and Literacy By 3 for the higher grades (n = 10). 

Specifically, most teacher respondents recommended ongoing or summer training in small groups, 
guided reading and other Literacy By 3 components to improve implementation. The training would 
support teachers who were hired during the year and who were not exposed to the initial summer 
professional development. 

They proposed other training including scheduling, literacy circles, yearlong comprehensive 
training, campus-based training in reading instruction, lesson planning, and effective use of the 
whole-reading block. 

Teachers also requested training in dual language models, exposure to lesson modeling and best 
practices that include sample lessons, videos, and demonstration lessons. Survey respondents 
wanted training to extend beyond the Literacy By 3 basics. 
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Teachers were also asked to propose additional information that they would like to see in the HISD 
curriculum guide to further support Literacy By 3 implementation. Using emergent themes, Figure 6 is a 
summary of the responses. The figure includes only those themes with 10 or more respondents. 
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Figure 6. Teacher perceptions and experiences with curriculum planning guides and Literacy By 
3, 2014-2015. 


e Most teacher respondents were either satisfied with the planning guides or did not offer any 
possible changes. 

e Teachers wanted more Literacy By 3 resources (n = 52) and more strategies and activities to 
be included in the planning guides (n = 46). 

e Some teachers expressed that the pacing guide was too long and should be shortened (n = 16) 
and, overall, the guide should be tailored to reflect the prekindergarten and kindergarten levels 
(n = 10). 

e Teachers indicated that the guides should be better aligned to Literacy By 3, the books, TEKS, 
and other aspects of the curriculum. 

e Some teacher respondents called for simplification of the guides (n = 13), and better alignment 
with Literacy By 3 (14). 


What were the effects of the Literacy By 3 Initiative on third- through fifth-grade students’ STAAR 
reading performance? 


Figure 7 and Figure 8 show the comparative STAAR mean reading scale scores by grade for students 
who had 2014 and 2015 test scores. Table 7 (Appendix A, page 30) shows the composition of the 
student-level sample by grade, gender, at-risk status, special education, G/T identification, ethnic group, 
economic status, and students’ LEP status. 
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*p < .001 (two-tailed) 


Figure 7. Comparative STAAR mean reading scale score of HISD third- to fifth-grade studenis in 
the sample, 2014 and 2015. 


e Figure 7 shows that the third-grade students in the sample had a higher mean scale score (M = 
1330.0, SD = 109.8) on the 2015 STAAR reading test compared to their mean scale score on 
the 2014 STAAR reading test (M = 1233.2, SD = 77.2). The difference was statistically 
significant, t (5618) = 22.6, p < .001 (two-tailed). The eta-squared statistic (.50) indicated a large 
and substantially important effect size (Table 8, Appendix A, page 31). These students were 
third-grade repeaters. 

e Fourth-grade students had a higher mean scale score (M = 1487.3, SD = 149.0) on the 2015 
STAAR reading test compared to their mean scale score on the 2014 STAAR reading test (M = 
1407.7, SD = 141.0) (Figure 9). The difference of 79.6 was statistically significant, t (14,217) = 
94.7, p< .001 (two-tailed). The eta-squared statistic (.29) indicated a moderate and substantially 
important effect size (Table 8, Appendix A, page 31). The Texas Education Agency (2015), has 
determined that for accountability purposes, individual fourth-grade students with a scale score 
difference of 82, over the same period, would have Met Progress on reading. 

e Fifth-grade students had a statistically significant higher mean scale score (M = 1538.8, SD = 
146.8) on the 2015 STAAR reading test compared to their mean scale score on the 2014 STAAR 
reading test (M = 1488.5, SD = 136.2) (Figure 9). The difference of 50.3 was statistically 
significant, t (13,298) = 61.2, p < .001 (two-tailed). The eta-squared statistic (.22) indicated an 
important effect size. (Table 8, Appendix A, page 31). The Texas Education Agency (2015), 
however, has determined that individual fifth-grade students with a scale score difference of 32, 
over the same period, would have Met Progress on reading. 

e Students in the third-grade sample had the highest mean reading scale score difference of 96.8 
scale score points, compared to fourth grade (79.6 scale score points) and fifth grade (50.3 
scale score points). 


Figure 8 displays comparative mean reading scale scores by ethnicity for the third-, fourth-, and 
fifth-grade students in the sample. 
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Figure 8. Comparative STAAR mean reading scale score by selected ethnicities for HISD 


third- to fifth-grade students in the sample, 2014 and 2015. 


African American, Hispanic, and White third-, fourth-, and fifth-grade students in the sample had 
higher mean scale scores on the 2015 STAAR reading test compared to their mean scale scores 
on the 2014 reading test. The differences in the scores for each of the three ethnic groups in the 
three grade levels were statistically significant, p < .001 or .05 (Table 9, Appendix A, page 31). 
Third-grade Hispanic students in the sample had the highest mean reading scale score 
difference (106.9 scale score points) followed by White students (104.3 points) and African 
American students (74.9 points) (Table 9). The largest effect was on Hispanic and White 
students with an eta-squared statistic of .57 in both cases indicating substantially important 
effect sizes (Table 9, Appendix A, page 31). 

In the fourth grade, White students had the highest mean reading scale score difference (105.8 
points) followed by African American (87.2 points) and Hispanic fourth-grade students (71.6 
scale score points) (Table 9). In the fourth grade, the largest effect was for African American 
students with an eta-squared statistic of 0.50 compared to 0.46 for White students and 0.33 for 
Hispanic students (Table 9, Appendix A, page 31). These three effect sizes were substantially 
important. The Texas Education Agency (2015), however, has determined that individual fourth- 
grade students with a scale score difference of 82, over the same period, would have Met 
Progress on reading. 

In the fifth-grade, White students had the highest mean reading scale score difference (78.2 
points) followed by African American (53.6 points) and Hispanic students (43.6 points) (Table 
9). Among the fifth-graders, the largest effect was seen for White students (eta-squared = 0.35) 
followed by African American students, (eta-squared = 0.28) and Hispanic students (eta- 
squared = 0.18) (Table 9, Appendix A, page 31). These eta-squared statistics were substantially 
important. 


The data in Figure 9 show the comparative mean scale score by the economic status of the third-, 


fourth- and fifth-grade students in the sample. 
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Figure 9. Comparative STAAR mean reading scale scores by economic status of third- to fifth- 


grade students in the sample, 2014 and 2015. 


Non-economically-disadvantaged third- through fifth-grade students had higher STAAR mean 
reading scale scores compared to their economically-disadvantaged peers (Figure 11). The 
fourth- and fifth-grade mean scale score differences between the two groups were statistically 
significant (p < .001) in favor of the non-economically-disadvantaged students (Table 10, 
Appendix A, page 32). The Texas Education Agency (2015), has determined that for 
accountability purposes, individual fourth-grade students with a scale score difference of 82, 
between 2014 and 2015 would have Met Progress on reading. The difference for a fifth-grade 
student would be 32 scale score points. 

In the third grade sample, the economically- disadvantaged students had a higher STAAR mean 
reading scale score growth (97.3 scale score points) compared to non-economically- 
disadvantaged students (89.7 points), with an eta-squared statistic of 0.51. The largest and 
substantially important effect was seen for the economically-disadvantaged third-grade students 
(Table 10, Appendix A, page 32). 


Figure 10 displays the comparative percentages of students, in the sample, who met Level Il 
Satisfactory phase-in 1 standards on the 2014 and the 2015 STAAR reading test by grade (English and 


Spanish). 
100.0 
® 80.0 
© 
e€ 60.0 
o 
5 40.0 
o 
20.0 
0.0 a 


Third Grade Fourth Grade Fifth Grade 
m2014 10.0 70.3 68.7 
m2015 52.4 65.6 71.8 


Figure 10. Comparative percentage of the third- through fifth-grade student sample who met 
Level II Satisfactory phase-in 1 STAAR reading standards, 2014 and 2015. 


HISD Research and Accountability 


15 


e The percentage of third-grade students in the sample who met Level II Satisfactory phase-in 
1 standard on the 2015 STAAR reading test increased from 10% in 2014 to 52.4% in 2015. 
This was an increase of 42.4 percentage points. Again, the 519 third-grade students in this 
sample were repeaters. 


e The proportion of fourth-grade students, in the sample, who met Level II Satisfactory phase- 
in 1 standard on STARR reading decreased from 70.3% in 2014 to 65.6% in 2015. This 
decreased by 4.4-percentage points. 

e In this study, the proportion of Literacy By 3 fifth-grade students who met Level II Satisfactory 
phase-in 1 standards on the 2015 STAAR reading test increased from 68.7% in 2014 to 71.8% 
in 2015. This was an increase of 3.1 percentage points. 

Figure 11 shows the comparative percentage of students who met Advanced performance standards 
on the 2015 STAAR reading test by grade level. 
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Figure 11. Comparative percentage of third- through fifth-grade students in the sample who met 
STAAR Advanced performance standard, 2014 and 2015. 


e The percentage of the third-grade student sample who met Advanced performance standards 
on the 2015 STAAR reading test increased from 0.2% in 2014 to 2.9% in 2015. This is an 
increase of 2.7 percentage points. These students were repeaters. 

e The percentage of fourth-grade students in the sample who met Advanced performance 
standards on the 2015 STAAR reading test increased from 17.0% in 2014 to 18.1% in 2015. 
This was an increase of 1.1 percentage points. 

e The percentage of fifth-grade students in the sample who met Advanced performance 
standards on the 2015 STAAR reading test increased from 16.7% to 20.9% in 2015. This was 
an increase of 4.2 percentage points. 


How did the Literacy By 3 first- through fifth-grade students perform on the lowa Assessments? 


Figure 12 shows the proportion of students who scored at or above the 50" National Percentile 
Rank (NPR) on the lowa Assessments. 
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Figure 12. Percentage of HISD Literacy By 3 first- through fifth-grade students who performed at 
or above the 50‘ NPR on the lowa Assessments ELA and reading, 2015. 


e The percentage of first- to fifth-grade students who scored at or above the 50° NPR on the 2015 
lowa ELA assessment ranged from 42.2% in the fifth grade to 53.3% in the first grade. 

e The percentage of students who scored at or above the 50 NPR on the 2015 lowa reading 
assessment ranged from 35.4% in the fourth grade to 53.9% in the first grade (Table 11, 
Appendix A, page 32). 

Discussion 


The purpose of this study was to assess the implementation, teacher experiences, and perceptions 
of the Literacy By 3 initiative and measure its effect on elementary student reading performance. The 
initiative focused on kindergarten- through third-grade teachers and students. This discussion focuses 
on preparation for implementation walkthrough visits, results of the teachers’ survey, and effects of the 
initiative on students reading and ELA performance. 


Preparation for implementation 

According to E-train data, 10,101 teachers and school officers completed training in five professional 
development components in preparation for implementing Literacy By 3 initiative. More than two-thirds 
of teachers completed training in each of the Literacy By professional development activities. Based on 
the survey results, they had at least some awareness of what the expectations were and how their 
classrooms should be organized to deliver the key components of Literacy By 3. This was also evident 
during the walkthrough of the selected classroom sample. Although each component was delivered in 
less than fourteen hours, the follow-up sessions could have helped to explain the positive effects of the 
initiative on students’ performance. 


Walkthrough 

Most of the fourteen classrooms visited had their rooms organized and arranged for Literacy By 3. 
They all had leveled books, book corners, arrangement for guided reading, and independent reading. 
Classrooms had evidence of a systematic approach to group assignment for guided reading. However, 
with few exceptions, teachers who were observed did not appear to use running records to assess 
students’ performance during the walkthroughs. Some schools used Accelerated Reading assessments 
to track students’ reading performance and most used Istation data. Most classrooms and even schools 
displayed some way of tracking students’ reading performance. During the debriefing sessions, 
principals in the high performing schools were confident, based on their experiences, that guided reading 
was the best approach to literacy instruction. 
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The two lower performing schools in the walkthrough sample (four classrooms) had other 
experiences that should be considered when analyzing literacy performance. They had higher 
proportions of African American students, higher student mobility rates, and slightly higher proportions 
of economically-disadvantaged students. Most teachers who were observed in these lower performing 
schools appeared to struggle with managing the rest of their classrooms during guided reading and to 
provide sufficient work to keep the rest of the class engaged in reading. There were no clear written 
outlines on the chalkboard of what students needed to do during the period compared to what was 
observed in the two high performing schools (four classes). Teachers in these high-performing schools 
had tasks written on the chalkboard. 

The high- and low-performing walkthrough schools had substantial differences in percentage of 
economically-disadvantaged students who met STAAR reading Level II Satisfactory phase-in 1 standard 
in 2015 (638% and 69% compared to 3% and 4% respectively). The high-performing schools observed 
appeared to have a culture of guided reading with both principals indicating during the debriefing that 
they had been using this practice for over ten years. 

From observation and through informal discussions, teachers appeared to place all students into 
appropriate groups for guided reading, rotate students by Literacy By 3 components on a regular basis, 
and reconstitute the groups based on assessment results from Istation, or Accelerated Reading, and 
running records. 


Survey 

Of the 701 (approximately 21.8%) of teachers who completed the Literacy By 3 survey, the results 
showed that 67.8% of them were implementing Literacy By 3 in the recommended 135 instructional 
minutes, however, almost one-third of the classrooms were not. One reason given was that reading 
instruction time had been restructured to 90 minutes in some schools. The research shows that reading 
is foundational to learning and failure to provide adequate exposure to the reading opportunities may 
affect students’ outcomes (Annie E. Casey Foundation, 2010). 

Teachers rated their Literacy By 3 practices (4.18 on a 5.0-point scale) much higher than material 
quality and usefulness, monitoring student growth and performance, and student reading practices. The 
training effect received average rating of 3.64 on a 5.0-point scale. None of the key aspects of the 
training received average ratings of 4.0 or higher on a 5.0-point scale. Taken collectively, if teachers 
completed multiple components, the training may have been adequate to affect students’ outcomes 
based on the U.S. Department of Education Institute of Education Sciences What Works Clearinghouse 
(Yoon, Duncan, Lee, Scarloss, & Shapley, 2009). Running records training and preparation information 
received during the four-day training had an average rating of 3.53, which may explain why so few 
teachers used them. Again, the running records are crucial for assessing and tracking students’ 
performance and for assigning students to guided reading groups. Based on the survey data, teachers 
requested further training in the use of running records in order to improve Literacy By 3 implementation 
(Figure 7). 

The support from campus leadership, the Literacy By 3 videos, and the January summit received 
the highest average ratings (3.71 — 3.75) for the training component. These data confirm the importance 
of providing year-round support for teachers during the implementation of the Literacy By 3. None of the 
average ratings associated with teacher perception and experiences with using HISD curriculum and 
instructional framework to guide Literacy By 3 were higher than 3.94 points. 

Teachers’ use of read-aloud protocols, use of anchor charts to support for instructional planning and 
delivery, and the 135-minute block itself as a structural provision for Literacy By 3 delivery received the 
highest ratings, between 3.80 and 3.94. In addition to training, teachers may need more structure and 
materials that allow them to plan, prepare, and execute the initiative’s key components. According to 
teacher respondents, materials should be appropriate and the existing curriculum may have to be 
revised to reflect Literacy By 3 approaches. In this respect, teachers called for more and sustained 
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training in small group organization and instruction, scheduling, modeling lessons, and best practices. It 
appears that they are willing to learn more and improve their performance through exposure to what 
works and how it works. The optimism must be built upon and principals and schools encouraged to 
make Literacy By 3 the primary approach to literacy instruction in all schools and particularly in 
underperforming schools. 


Program effect 

Results from the repeated-measures design indicated that the 2015 STAAR mean reading scale 
scores were significantly higher for the third-, fourth-, and fifth-grade students after the year of Literacy 
By 3 than before this initiative. While the fourth grade students showed a scale-score increase of 79.6 
over their third-grade performance, the Texas Education Agency (2015) has determined that for 
accountability purposes, individual fourth-grade students with a scale score difference of 82 between 
2014 and 2015 would have Met Progress on reading. The average scale score increase of 50.3 for fifth 
grade students exceeded the State’s standard of 32 scale-score points for Met Progress in reading for 
individual students. The proportion of students who met Level II phase-in 1 Satisfactory reading 
standards was also higher after the Literacy By 3 initiative except in the fourth-grade. 

The Literacy By 3 initiative had substantially important effects on the third grade (eta-squared = 
0.50) and fourth grade (eta-squared = 0.39) and important effect on fifth grade (eta-squared = 0.22) 
reading based on the Texas Education Agency Best Practices Clearinghouse (TEABPC) effect size 
benchmark (See Texas Education Agency, 2011). Based on Fryer’s (2012) benchmark of .08 standard 
deviations as equal to one month of improvement, these effect sizes indicate approximate growth of 
between three to six months. This is, particularly, significant for the third grade sample since these 
constituted students who were retained in the grade but who, on average, showed a six-month growth 
in reading. 

When disaggregated by ethnicity, Literacy By 3 probably explained between 18% and 57% in the 
variance in the reading scale scores between 2014 and 2015. This is equivalent to approximately two- 
to seven-months growth in reading. Third-grade Hispanic and White students had the largest average 
reading growth, equivalent to approximately seven months growth (57% of the variance) and fifth-grade 
Hispanic students had the smallest average reading growth (approximately 2 months). Fourth-grade 
African American students demonstrated an average reading growth of six months. 

When disaggregated by student economic status, Literacy By 3 had important to substantially- 
important effects on both groups ranging from an eta-squared statistic of 0.16 through 0.51 in the third, 
fourth, and fifth grades. The initiative probably explained between 16% and 51% of the variance in the 
reading scale scores. This is equivalent to approximately two to six months growth in reading after one 
year’s Literacy By 3 based on Fryer’s (2012) benchmark. 

The lowa Assessments are in their initial administration in HISD; therefore, there are limits to the 
analysis for first- through-fifth grade students. Only in the first grade did the majority of students perform 
at or above the 50" NPR for both ELA Total and reading. A majority of fourth-grade students performed 
at or above the 50'" NPR for ELA Total. 

In summary, Literacy By 3 resulted in substantial improvement in students’ reading performance, 
including low-performing students. Students were exposed to other programs but supplementary to 
Literacy By 3. After the literacy interventions, a higher proportion of students, in the sample, met the 
STAAR Level Il phase-in 1 Satisfactory reading standards at the third, and fifth-grades and Advance 
performance standards at all three grade levels. 

Based on the survey results and walkthrough debriefings, principals and teachers recognize the 
challenges associated with implementing Literacy By 3, organizing and managing guided reading 
groups, reorganizing the curriculum, accessing adequate materials, and assessing students’ 
performance and progress. They also identified its merits in meeting the reading needs of students in 
small groups who may otherwise face challenges in whole class instruction. While teachers rated their 
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own instructional practices higher, they appeared to agree that sustained support is essential. Analyses 
showed that after one year, the students, on average, showed a two- to seven-month growth in reading 
in the third, fourth, and fifth grades in comparison to the year, prior. The improvement and effects were 
substantive for minority, underperforming, and economically-disadvantaged HISD student subgroups 


Recommendations 


e Consistent with respondents’ recommendations, the district should continue training in small 
group organization and instruction, guided reading, instructional strategies, and running records 
for Literacy By 3 in order to improve classroom practices. 

e Continued training in the management of small or guided reading groups should focus on how 
to keep students engaged, how to pace work assignment for students, and exposing teachers 
to exemplars and Literacy By 3 best practices. 

e Leveled books and related materials for all, and in particular LEP classrooms, need to be 
available to support instruction. 

e Students should be exposed to the required 135 minutes for Literacy By 3 instruction. 
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Appendix A 


Table 1. Summary of walkthroughs for a sample of HISD Literacy By 3 schools, first and third grade, 2015. 
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Table 1. Continued 
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Note: AB1, BC33, CD1MS and so on are unique classroom identifiers. 
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Table 2. Distribution by programs of HISD teachers and school leaders who participated in the Literacy By 3 professional 


development sessions, 2014-2015. 


Literacy by 3 Professional development No. of sessions Hours of Completed Enrolled Incomplete No show 
program PD per 

session 
ABC’s of guided reading 17 12 1,833 185 127 309 4,415 
ABC’s of guided reading day 1/day 2 7 12 1,939 9 0 13 1,961 
Independent reading 18 6 2,667 152 16 227 3,111 
Read aloud 16 6 2,012 176 8 337 2,533 
Total 58 8,451 522 151 928 12,062 


Figures exclude prek and k 
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TABLE 3. SOCIOECONOMIC, DEMOGRAPHIC, AND ACADEMIC CHARACTERISTICS OF LITERACY BY 3 WALKTHROUGH SCHOOLS, 


PAU 0 
SCHOOL Expenditure Per Ethnic Composition (%) Socioeconomic status (%) STAAR Reading Performance (%) TEA Accountability Rating 
Student ($) 
Instruction African Hispanic White Econ. ‘At ELL Mobility Met Level Il Econ. Disadv. Met Econ. Disadv. 2014 Ratings 
American (%) (%) Disadv. Risk (%)* phase-in 1 Who Met Level Il Advanced Who Met 
(%) Satisfactory phase-in 1 Standards Advanced 
Standard (%) Satisfactory (%) Standards (%) 
Standard (%) 
LP2015 4,573 We) 21.6 0.3 96.5 86 16.0 22.6 38 3 37 3 Improvement Required 
AP2015 4,520 1.8 95.6 1.8 94.3 83 46.6 9.5 71 15 70 13 Met Standards 
SE2015 4,838 0.2 99.2 0.6 95.0 78 54.4 12.0 80 15 79 14 Met Standards 
ME2015 4,592 92.0 6.6 0.7 99.1 87 3.7 31.7 48 4 47 5 Improvement Required 
VE2015 4,037 0.8 98.0 0.6 94.5 83 47.3 6.4 79 39 78 39 Met Standards 
HI02015 4,525 6.5 16.3 45.2 10.1 20 5.5 3.2 98 63 100 43 Met Standards - Distinction 
1HO2015 4,256 1.6 36.7 66.7 1.8 16 1.8 Dal 99 69 100 55 Met Standards — Distinction 
Sources: Texas Education Agency, 2013-14 School Report Card *These are only for 3- to 5""-grade rates. 


HISD, District and School Profiles, 2013-2014 


Note: LP2015, AP2015, and so on, are unique school identifiers. 


Notes: 

Of the seven schools where walkthroughs were conducted, two had improvement required (IR) status on the 2014 state accountability system; two met state standards with distinctions 
on reading/ELA, math, science, student progress, closing the achievement gap, and students’ postsecondary readiness, and three met standards. State standards are determined using 
weighted scores on student achievement, student progress, closing the achievement gap, and postsecondary readiness. 
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The walkthrough IR schools had the highest proportion of African American students (77.9% and 92.0%), the highest proportion of at-risk students (86% and 87%), and the highest 
proportion of economically disadvantaged students (99.1% and 96.5%). They also had the highest student mobility rate for grades three to five (31.7% and 22.6%), the lowest percentage 
of students who met Level Il, phase-in 1 Satisfactory reading standard as well as advanced standards (38% and 48%). In addition, they had the lowest percentage of economically- 
disadvantaged students who met STAAR Level II phase-in 1 Satisfactory reading standard (3% and 4%), and Advanced reading standards (3% and 5%). 

The walkthrough schools that only met state standards had the highest proportion of Hispanic students (95.6%, 98.0% and 99.2%), the largest percentage of ELL students (46.6%, 
54.4% and 47.3%). These schools also had relatively high proportions of economically-disadvantaged students (94.3%, 95.0%, and 94.5%). Between 71 and 80% of their students met 
STAAR Level Il phase-in 1 Satisfactory reading standard while between 15% and 39% of the economically disadvantaged met the state standard and 13-39% met the Advanced reading 
standard. 

Those walkthrough Literacy By 3 schools that met standards with distinction had the highest percentage of White students (45.2% and 66.7%). Those schools had the lowest proportion 
of economically- disadvantaged, at-risk, ELL and mobile students. They had also the highest percentage of students (98% and 99%) who met the Level II phase-in 1 Satisfactory reading 
standard level with 100% of the economically- disadvantaged students meeting STAAR Level II phase-in 1 Satisfactory reading standard. Of these economically-disadvantaged students, 
43% and 55%, respectively, met the Advanced reading standards. 
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Table 4: Teacher experiences and perceptions on Literacy By 3 materials, practices, and student 


eXe\acolgunr-lavex-Miaceyalixe)alarem-larema=y-Cel (ate m el-al-hUCe) ean | be) Dle-0h tot hoe 


LITERACY 
FACTORS 


TRAINING EFFECT 


MATERIAL 
QUALITY AND 
USEFULNESS 


TEACHER 
LITERACY 
PRACTICES 


MONITORING 
STUDENTS 
PERFORMANCE 
AND GROWTH 


STUDENT 
PRACTICES 


Answered question 


Skipped question 


Please rate your degree of agreement or disagreement with the following statements: 


Answer Options 


In comparison to one year ago, | 
have a deeper understanding of 
what it means to be an effective 
literacy teacher. 

The Scholastic leveled reading 
materials and resources 
available to my classroom are 
high quality. 

The Scholastic leveled reading 
materials and resources 
available to my classroom are 
useful. 

| facilitate interactive read aloud 
on a daily basis in my 
classroom. 

The identified English/Spanish 
Language Arts TEKS are 
practiced as a common thread 
throughout my daily literacy 
block. 

| facilitate small group guided 
reading instruction daily in my 
classroom. 

Students read books on their 
independent level during 
independent reading time daily 
in my classroom. 

| use running records to form 
guided reading groups. 


| use running records to assess 
student-reading levels. 


| use running records to monitor 
reading growth. 


| use Istation to form guided 
reading groups. 


| use Istation to assess students’ 
reading levels. 


| use Istation to monitor reading 
growth. 


Students in my classroom 
independently select a "just right 
book" for independent reading. 
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Agree 
(%) 
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43.6 


42.8 
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28.7 


29.4 


30.1 


30.5 


31.2 


31.5 


36.0 


Rating 
Average 


3.65 


3.74 


3.83 


4.23 


4.22 


4.12 


417 


3.72 


3.74 


3.76 


3.70 


3.72 


3.77 


4.00 


Response 
Count 
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649 
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44 
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Table 5. Teacher experiences and perceptions on their Literacy By 3 training, HISD, 2014— 


2015. 


PLEASE RATE YOUR DEGREE OF AGREEMENT OR DISAGREEMENT WITH THE FOLLOWING STATEMENTS: 


ANSWER OPTIONS 


The information | learned during the 
four-day training prepared me to begin 
implementing Literacy By 3 in my 
classroom. 

The early release training on small 
group guided reading instruction 
provided learning that helped me better 
plan and deliver small group instruction 
to my students. 

The running record training provided an 
opportunity to learn and practice a 
running record. 


The running record training provided an 
opportunity to understand how to move 
students’ reading levels. 


The January 5th pk-5 literacy summit 
provided me with the literacy instruction 
practices that | could implement in my 
classroom. 

The HISD Literacy By 3 video resources 
were a useful model for good 
instructional practices for literacy 
instruction. 

The Literacy By 3 leaders on my 
campus provide value support and 
information about best literacy 
practices. 

Answered question 


Skipped question 


Strongly 
Disagree 
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Disagree 
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Agree 
(%) 


18.6 


17.7 


18.1 


18.6 


eS 
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15.4 


Agree 
(%) 
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Strongly 
Agree 
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17.5 
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12.9 


14.3 
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N/A 
(%) 


14.24 


31.4 


29.8 


30.6 


48.9 


23.7 


Rating 
Average 


3.53 


3.53 


3.57 


3.53 


3.72 


3.71 


3.75 


Response 


Count 


639 


637 


637 


634 


636 


637 


638 


640 


61 
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guides and protocol for the delivery of Literacy By 3 initiative. 


Answer options 


The structure and framework of the HISD 
elementary reading curriculum planning 
guides facilitate the ease of planning. 


The "first 25 days" document supported 
my implementation of literacy by 3. 


The 135-minute reading block provides 
structure to reading instruction in my 
classroom. 


l use read aloud protocols outlined in the 
HISD curriculum planning guides. 


The instructional strategies outlined in 
the HISD curriculum planning guides 
provide support for my planning and 
delivery of instruction. 


The anchor charts outlined in the HISD 
curriculum planning guides provide 
support for my planning and delivery of 
instruction. 


Answered question 


Skipped question 


Strongly 
Disagree 


8.4 


6.3 


4.4 


3.3 


4.9 


4.8 


Disagree 


Ly 


8.7 


41 


3.5 


5.5 
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Please indicate your level of agreement or disagreement with the following statements: 


Somewhat 
Agree 


22.8 


20.3 


17 


17 


16.6 


Agree 
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30.5 


40.2 


41.5 


38.7 


36.9 


Strongly 
Agree 


16.2 


25.2 


27.3 


28.8 


26.9 


31.5 


N/A 


4.9 


5.9 


4.3 


5.3 


Rating 
Average 


3.46 


3.65 


3.88 


3.95 


3.81 


3.90 


Response 
Count 


631 


632 


630 


631 


633 


628 


634 
67 
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Literacy By 3 study sample. 


Third Fourth Fifth 
Sample Size 519 14,299 13,299 
Male 58.8 49.6 49.4 
Female 41.2 50.4 50.6 
African American 31.6 23.1 22.4 
Hispanic 66.7 64.9 65.2 
White il.2 7.5 7.8 
Non-Special Education 93.8 96.0 96.8 
Special Education 5.8 3.8 3.1 
Non-At-Risk 0.6 35.2 38 
At-Risk 99.4 64.7 61.9 
Non-G/T 98.8 75.9 72.1 
G/T 0.8 23.9 27.8 
Non-Economically Disadvantaged 6.4 20.1 20.9 
Economically Disadvantaged 93.6 79.9 79.1 
Non-LEP 52.4 52.6 54.2 
LEP 47.6 47.3 45.7 
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Table 8. Results of paired t-test and effect sizes of Literacy By 3 initiative on HISD third-, fourth-, 


Varo Mititnomele-(o(- Mol WAV a @c-r-lol late (21) a0 oot bo 


Sample Standard Mean Significance (2- Effect Size (Eta 
Deviation Difference tailed) squared) 

Third 2015 519 1330.0 109.8 22.6 96.8 0.00* 0.50 
2014 519 1233.2 rye 

Fourth 2015 14,218 1487.3 149.0 94.7 79.6 0.00* 0.39 
2014 14,218 1407.7 141.0 

Fifth 2015 13,299 1538.8 146.8 61.2 50.3 0.00* 0.22 
2014 13,299 1488.5 136.2 

*p < .001 


Table 9. Results of paired t-test and Literacy By 3 effects on HISD third-, fourth-, and fifth-Grade 


Major Ethnic Groups’ STAAR Reading Test, 2014-2015. 


Ethnicity Year Sig (2- Effect Size 
tailed) (Eta 
squared) 
Third 2015 1310.8 118.9 74.9 9.2 0.00* 0.26 
African 
Ameriegn 2014 tel 1235.9 74.2 
2015 1339.3 104.9 106.9 21.2. 0.00* 0.57 
Hispanic 
2014 346 1232.4 78.9 
2015 1316.5 63.3 104.3 2.6 0.05** 0.57 
White 
2014 6 1212.2 46.5 
Fourth 2015 1456.9 134.3 87.2 56.9 0.00* 0.50 
African 
American 2014 3,248 1369.6 126.3 
2015 1471.0 137.6 71.6 67.8 0.00* 0.33 
Hispanic 
2014 9,236 1399.3 135.6 
2015 1639.3 152.3 105.8 30.1 0.00* 0.46 
White 
2014 1,064 1533.5 136.9 
Fifth 2015 1507.3 134.2 53.6 33.8 0.00* 0.28 
African 
pneees 2014 an 1453.7 123.3 
2015 1522.2 134.1 43.6 43.1 0.00* 0.18 
Hispanic 
2014 8,669 1478.6 129.9 
2015 1688.2 149.7 78.2 23.7 0.00* 0.35 
White 
2014 1,042 1610.0 138 


*p < .001 (two-tailed) 
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**p < .05 (two-tailed) 
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Table 10. Results of paired t-test results and Literacy By 3 effect sizes on HISD third-, fourth-, 
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Grade Econ Status Year n Mean SD MD t Sig (2- Effect 
tailed) Size (Eta 
squared) 
Third 2015 1324.7 146.9 89.7 4.0 0.00 0.33 
Non-Economically 
Disadvantaged 2014 33 1235.0 93.5 
2015 1330.0 107.0 97.3 22.5 0.00 0.51 
Economically 
Disadvantaged 2014 486 1233.0 761 
Fourth 2015 1589.6 159.2 100.7 52.8 0.00 0.49 
Non-Economically 
Disadvantaged 2014 2,856 1488.8 142.6 
2015 1461.6 134.7 74.3 79.9 0.00 0.36 
Economically 
Disadvantaged 2014 11,371 1387.3 133.4 
Fifth 2015 1642.5 158.6 70.7 36.2 0.00 0.32 
Non-Economically 
Disadvantaged 2014 20tT 1571.4 442.5 
2015 1511.4 130.5 44.9 50.3 0.00 0.16 
Economically 
Disadvantaged 2014 10,521 1466.5 1256 


*p < .001 (two-tailed) 


Table 11. lowa ELA and reading NCEs, percent performing at or above the 50 NPR and mean 
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Mean NCE 50 NPR Mean Standard Score % Performing At or Above the 50th 
Standard Score NPR 
Grade n ELA Reading ELA & Reading ELA Reading ELA Reading 
First 11,782 52 50 150 150.3 151.5 53.3 53.9 
Second 11,956 48 46 168 166.9 168.1 46.9 43.2 
Third 12,520 47 44 170 186.6 179.9 43.8 36.1 
Fourth 14,850 48 43 200 204.9 193.3 45.0 35.4 
Fifth 15,270 47 44 214 215.2 205.7 42.2 36.3 
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